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Abstract. We prove that a Schur function of rectangular shape (M") whose variables are 
specialized to xi, , . . . ,Xn, x^ factorizes into a product of two odd orthogonal characters 
of rectangular shape, one of which is evaluated at — a;i, . . . , —Xn, if M is even, while it factorizes 
into a product of a symplectic character and an even orthogonal character, both of rectangular 
shape, if M is odd. It is furthermore shown that the first factorization implies a factorization 
theorem for rhombus tilings of a hexagon, which has an equivalent formulation in terms of 
plane partitions. A similar factorization theorem is proven for the sum of two Schur functions 
of respective rectangular shapes (M") and (M"^^). 



1. Introduction. The purpose of this note is to prove curious factorization properties for 
Schur functions of rectangular shape, which seem to have escaped the attention of previous 
authors. (We refer the reader to Section 2 for all definitions.) More precisely, we show that 
a Schur function of rectangular shape (M"^) which is evaluated at xi,X2, ■ ■ ■ ^Xn and their 
reciprocals x^^, x^^, . . . , x~^ factorizes into two factors, and the same is true for the sum of 
two Schur functions of respective shapes (M") and (M"~^). 
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We begin by describing explicitly the case of one Schur function. If M is even, then 
both factors are odd orthogonal characters of rectangular shape, one of them evaluated at 
the variables xi,X2, ■ ■ ■ ,Xn, but the other is evaluated at —xi, —X2, • • • , —Xn- If M is odd, 
then one factor is a symplectic character of rectangular shape, while the other is an even 
orthogonal character of rectangular shape, both being evaluated at xi,X2, ■ ■ ■ , Xn- The case 
of even M of this factorization property is presented in the following theorem. 

Theorem 1. For any non-negative integers m and n, we have 

^((2m)") (^1) ^1 -1X21X2 , ■ . . 1 Xn, X^^) 

= (-l)'^"so(^«)(a;i,a;2,...,a;n) so(^«)(-a;i, -0:2, • • • , -a^n). (1-1) 

If M is odd, then the factorization takes the following form. 
Theorem 2. For any non-negative integers m and n, we have 

S((2m+l)")(a^l, X^^, X2, X2^, • • • , O^n, X~^) 

= SP(^„)(a;i, X2, ...,Xn) ol^^_^^^r.){xi,X2, . . .,Xn). (1.2) 

Since our identities involve classical group characters, one might ask whether there are 
representation-theoretic interpretations of these identities. At first sight, this seems to be a 
difficult question because of the somewhat "incoherent" right-hand sides of (1.1) and (1.2). 
However, there is a uniform way of writing the factorization identities of Theorems 1 and 2 
that was pointed out to us by Soichi Okada. Namely, by comparing (2.2) and (2.6), and by 
using (2.3) and (2.7), we see that 

AT 

(_l)Ef=i A, sox{-xi, -X2, -xn) Hix^ + xT^/'') = of+«|(a;i, X2...,xn), (1-3) 

where A -|- | is short for (Ai -|- ^, A2 + |, . . . , Ajv + |)- Furthermore, by comparing (2.2) and 
(2.4), and by using (2.3) and (2.5), we see that 

JV 

{Xi +Xi ^ ) = so^+i{xi,X2...,xn)- (1.4) 

Theorem 1 and 2 may therefore be uniformly stated as 
f TJ( I / -1 -1 -l^ 

I J_J_(,2^i + ) I '5(M")l^l7 ^1 ,X2,X2 , . . . ,Xn, X^ ) 

= *^((M)r.) (^1' a^2, • • • , Xn) 0^^"M+i)„j (-^1, 3^2, • • • , Xn). (1-5) 

Written in this way, this may, in the end, lead to a representation-theoretic interpretation of 
these identities, although we must confess that we are not able to offer such an interpretation. 
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On the other hand, we are able to offer a combinatorial interpretation for Theorem 1. 
As we show in Section 5, if we speciahze xi = X2 = • ■ ■ = Xn = 1 it^ Theorem 1, then one 
obtains a factorization theorem for rhombus tihngs of a hexagon, which has also a natural, 
equivalent formulation as a factorization theorem for plane partitions (see (5.1)). It is, in 
fact, this factorization theorem which we observed first, and which formed the starting point 
of this work. We suspect that a more general factorization theorem for rhombus tilings is 
lurking behind. If there is a natural combinatorial interpretation of Theorem 2 is less clear. 
We make an attempt, also in Section 5, but we consider it not entirely satisfactory. 

The case of the sum of two Schur functions of rectangular shapes is treated in Section 6. 
Namely, we show that there are very similar factorization theorems for the sum of two Schur 
functions of respective rectangular shapes (M"^) and (M"^~^) (see Theorems 3 and 4). The 
existence of these was pointed out to us by Ron King. 

The proofs of Theorems 1 and 2 arc given in Section 4. These proofs are based on an 
auxiliary identity which is established in Section 3 (see Lemma 1). The proofs of Theorems 3 
and 4 are based on two further auxiliary identities of similar type, which are also presented 
and proved in Section 3 (see Lemmas 2 and 3). 

2. Classical group characters. In this section we recall the definitions of the classical 
group characters involved in the factorizations in Theorems 1 and 2. We also briefiy touch 
upon their significance in representation theory. 

Given a partition A = (Ai, A2, . . . , Ajv) (i-e., a non-increasing sequence of non- negative 
integers) the Schur function sx{xi,X2, ■ ■ ■ ,xn) is defined by (see [4, p. 403, (A. 4)], [7, 
Prop. 1.4.4], or [8, Ch. I, (3.1)]) 



det (0:^+^-*) 
..,x„)= . (2.1) 

l<h,t<N 



It is not difficult to see that the denominator in (2.1) cancels out, so that any Schur function 
sx{xi, X2, . . . , .Tjv) is in fact a polynomial in xi, X2, . . . ,xn, and is thus well-defined for any 
choice of the variables xi, X2, . . . ,xn- It is well-known (cf. [4, §24.2]) that sx{xi,X2, . . . ,xn) 
is an irreducible character of SLn{C) (respectively GLn{C)). 

Given a non-increasing sequence A = (Ai, A2, . . . , A„) of integers or half-integers (the 
latter being, by definition, positive odd integers divided by 2), the odd orthogonal character 
sox{xi,X2, . . . ,xn) is defined by (see [4, (24.28)]) 



/ X l<h,t<N , , 

sox{xu 0:2, ... , xn) = -(iv-t+i), • (2.2) 

det {xr^ ^^h ) 

l<h,t<N 



Again, it is not difficult to see that the denominator in (2.2) cancels out, so that any odd 
orthogonal character 50^(^1, X2, ■ ■ ■ ,xn) is in fact a Laurent polynomial in xi,X2, ■ ■ ■ ,xn 
(i.e., a polynomial in xi, Xi^,X2, X2^, . . . ,xn, x^^), and is thus well-defined for any choice 
of the variables xi,X2, ■ ■ ■ , xn such that all of them are non-zero. By the Weyl denominator 
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formula for type B (cf. [4, Lemma 24.3, Ex. A.62]), 



det (xr^'^-x;^''-'-'^^) 

l<h,t<n \ ^ ) 



= (xiXs-'-Xn) JJ {xh-Xt){xhXt-\)W{xy,-\), (2.3) 

l<h<t<n h=l 

the denominator in (2.2) can be actually evaluated in product form. It is well-known (cf. [4, 
§24.2]) that sox{xi, X2, . . . ,xn) is an irreducible character of S02n+i{C). 

Given a partition A = (Ai, A2, . . . , A„), the symplectic character sp)^{xi,X2i ■ ■ ■ iXjsi) is 
defined by (see [4, (24.18)]) 



det {x';,'^''-'^' -x-'^'^''-'^'>) 



spxixi, X2,..., xn) = ' - -(N-t+iu • (2-4) 

det [Xf^ -Xf^^ ') 

l<h,t<N 

Similarly to odd orthogonal characters, sp^^{xi, X2, . . . ,xn) is a, Laurent polynomial in xi, X2, 
. . . , Xn, and is thus well-defined for any choice of the variables xi, X2, • • • , xn such that all 
of them are non-zero. By the Weyl denominator formula for type C (cf. [4, Lemma 24.3, 
Ex. A.52]), 

n 

dei {xl-'+'-xl-'+') = {x,---Xr.)-- n {xh-Xt){xHXt-l)\[{xl-l\ (2.5) 

l<h,t<n 

~ ~ l<h<t<n h=l 

the denominator in (2.4) can be actually evaluated in product form. Furthermore, 
spx{xi, X2j . . . , a;jv) is an irreducible character of S'p2Af (C) (cf. [4, §24.2]). 

Finally, given a non-increasing sequence A = (Ai, A2, . . . , Atv) of positive integers or half- 
integers, the even orthogonal character o^^^{xi,X2i . . . ,xn) is given by (see [4, (24.40) plus 
the remarks on top of page 411]) 

det (xi'+''-'+.-<^'+"'-*>) 

or"(x„x„...,x«) = 2 '-''''-^ _ ■ (2.6) 

det Ix, +Xf,^ M 

l<h,t<N 

Here as well, o'^^'^{xi, X2, ■ ■ ■ , xn) is a Laurent polynomial in xi,X2, ■ ■ ■ ,xn, and is thus 
well-defined for any choice of the variables xi,X2, ■ ■ ■ ,xn such that all of them are non-zero, 
the Weyl denominator formula for type D (cf. [4, Lemma 24.3, Ex. A. 66]), 



det (xr*+x-("-*)) = 2-(a;i---xO-+i TT (x^ - x,)(x^x, - 1), (2.7) 

l<h,t<n 

l<h<t<n 



the denominator in (2.6) can be again evaluated in product form. It is well-known (cf. [4, 
§24.2]) that o^^'^"(a;i, a;2, . . . ,xn) is an irreducible character of 02n{C). When restricted to 
S02n{C), it splits into two different irreducible characters of S02n{C). (The reader should 
observe that we assumed Ajv > 0. If we had allowed Ajv = 0, then we would have to divide 
the right-hand side of (2.6) by 2 in order to obtain an irreducible character of 02jv(C) or 
S02n{C).) 
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3. Auxiliary identities. The proofs of Theorems 1 and 2, and those of Theorems 3 and 
4 in Section 6, hinge upon certain multivariable identities which we prove in this section. 

Before we are able to state these identities, we have to introduce some notation. 

Throughout, we use the standard notation [n] := {1,2, . . .,n}. Let A and B be given 
subsets of the set of positive integers. Slightly abusing notation for resultants from [7], we 
define 

(In order to avoid any confusion on the part of the reader: the symbol in R{A, B~^) 
has no meaning by itself, the exponent —1 is just there to indicate that the reciprocals of 
the variables indexed by B are used on the right-hand side of the definition.) Furthermore, 
we define 

V{A) := J] [xa - Xb) 

a,beA 
a<b 

and 

a,beA 
a<b 

In all these definitions, empty products have to be interpreted as 1. 

Now we are in the position to state and prove the identity on which the proofs of Theo- 
rems 1 and 2 are based. 

Lemma 1. For all positive integers N, there holds the identity 
J2 V{A)V{A-^) ViA'')V{iA''y^) R{A, A'^) R{A'', (A'')-^) 

AC [2 AT] 
|A|=iV 

= V{A)V{A-^)V{A'')V{{A'')-^)R{A,{A'')-^) R{A'',A-^) , (3.1) 

AC[2N] 

where A'^ denotes the complement of A in [^N] . 

Proof. We prove the assertion by induction on N . For A*" = 1, identity (3.1) reduces to 

2{xi — X^^){X2 — X2^) = 2(xi — X2){x^^ — X2^) + 2(xi — X2^)(X2 — X^^), 

which can be readily verified. 

Now let us suppose that we have proved (3.1) with N replaced by A*" — 1. Below, we 
shall show that this implies that the identity (3.1) is true if we specialize xi to X2. Let 
us for the moment suppose that this is already done. Since, as is easy to see, both sides 
of (3.1) are symmetric in the variables xi, X2, . . . , X2n, as well as they remain invariant up 
to an overall multiplicative sign if we replace xi by (on the left-hand side of (3.1), 
the latter assertion already applies to each individual summand; on the right-hand side. 
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one has to group the summands in pairs: the summands corresponding to a set A and the 
symmetric difference ^A{1} have to be considered together), this means that identity (3.1) 
holds for all speciahzations of xi to one of X2, ^, a^s, ^Cg ^, . . . , X2n, ^2^. These are — 2 
speciahzations. On the other hand, as Laurent polynomials in xi, the degree of both sides 
of (3.1) is at most 2N — 1. (That is, the maximal exponent e of a power xf is e — 2N — 1, 
and the minimal exponent is e = —{2N — 1).) Hence, both sides of (3.1) must agree up to 
a multiplicative constant. In order to determine this multiplicative constant, we compare 
coefficients of 

2Ar-l 2iV-3.. 1 .^2N-12N-3_1 

on both sides of (3.1). On the left-hand side, the only terms contributing are the ones corre- 
sponding to ^ = {1, 2, . . . , N} and A = {N + 1, + 2, . . . , 2A'"} (in fact, the corresponding 
terms are equal to each other), both of them contributing a coefficient of 1. On the right- 
hand side, the only terms contributing are again the ones corresponding toA = {l,2,..., N} 
and A — {N + 1, N + 2, . . . , 2N} (being equal to each other), both of them also contributing 
a coefficient of 1. 

It remains to prove that, under the induction hypothesis. Equation (3.1) holds for xi = X2- 
Indeed, under this specialization, terms corresponding to sets A which contain both 1 and 
2 and to those which contain neither 1 nor 2 vanish on both sides of (3.1), because of the 
appearance of the Vandermonde products V{A) respectively V{A'^). Therefore, if xi = X2, 
identity (3.1) reduces to 



f^^ - - - - \ 

(xi - X^ ^)^f Ylixi - Xj){x^ ^ - X- ^){xi - X- ^){Xj -x^^)\ 



X V{A)V{A-^)V{A^)V{{A^)-^)R{A,A-^)R{A^,{A^)-^) 

A(Z{Z,A,...,2N} 
\A\=N-1 

= {Xi - X^ i Ylixi - Xj){x^ ^ - Xj ^){XI - Xj ^){Xj -x^^)j 

X J2 V{A)V{A-^)V{A'')V{{A'')-^)R{A,{A'')-^)R{A^,A-^) 

AC{3,4,...,2iV} 



After clearing the product which is common to both sides, we see that the remaining identity 
is equivalent to (3.1) with N replaced by AT — 1, the latter being true due to the induction 
hypothesis. This finishes the proof of the lemma. □ 

On the other hand, the proofs of Theorems 3 and 4 are based on the following two lemmas. 
The reader should observe the subtle difference on the right-hand sides of (3.1) and (3.2) 
(the left-hand sides being identical). 
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Lemma 2. For all positive integers N , there holds the identity 
J] V{A)V{A-^) V{A'')V{{A^)-'') R{A, A'^) (vl')-') 

AC [2 AT] 
|A|=iV 

= ^-^:>d^^V{A)V{A-^)V{A'')V{{A^)-^)R{A,{A'')-^)R{A'',A-^), (3.2) 

A<Z[2N] 

where A^ denotes the complement of A in [^N], and where x~"^ is short for noeA-^o^ ^'^'^ 
x^'' is short for HaeA^ ^a- 

Proof. We proceed as in the proof of Lemma 1. That is, we perform an induction on N . 
For N = 1, identity (3.2) reduces to 

2(xi — Xi^){x2 — ^2^) — XiX2{xi — X2){Xi^ — X2^) + XiX2^{xi — X2^){x2 — Xi^) 

+ X^^X2{X2 — X^^){xi — X2^) + X^^X2^{x^^ — X2^){xi — X2), 

which can be readily verified. Almost all the remaining steps are identical with those in the 
proof of Lemma 1, except that more care is needed to show that, as a Laurent polynomial 
in xi, the degree of the right-hand side of (3.2) is at most 2N — 1. Indeed, by inspection, 
this degree is at most 2A'", and the coefficient of x\^ is equal to 

^ {-1)^'^^\-W{A)V{A-^) V{A'')V{{A'')-^) R{A, {A")-^) R{A'',A-^) , 

AC[2iV]\{l} 

where A'^ now denotes the complement of A in [2iV]\{l}. (Note that only those subsets 
A of [2A^] contribute to the coefficient of xf^ on the right-hand side of (3.2) which do not 
contain 1, whence the term x~'^x^ on the right-hand side of (3.2) got cancelled due to the 
contributions from the terms R (A, (A*^)"^) and V ((A^)~^), respectively.) However, in this 
sum, the terms indexed by A respectively A^ cancel each other, so that this sum does indeed 
vanish. □ 

Lemma 3. For all positive integers N, there holds the identity 
J2 V{A)V{A-^) V{A'')V{{A'')-^) R{A,A-^) R{A'', {A")-^) 

AC[2N+1] 

\A\=N 

= J2 (-l)^+'^'x-^x^V(A)V(A-i) V(A^) V((A^)-^) R{A, {A")-^) A-^) , 

AC[2Ar+l] (3.3) 

where A^ denotes the complement of A in [2Ar -|- 1], while is short for Yla&A^a^ ^'^^ 
x"^" is short for YlaeA'^ ^a, cls before. 

Proof. We proceed again as in the proof of Lemma 1. Here, the induction basis, the case 
TV = of (3.3), reads 

{xi — Xi^) — Xi — Xi^. 
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As Laurent polynomials in xi, the degree of the left-hand and right-hand sides of (3.3) are at 
most 2N + 1. This means that we need 4A'"-|- 3 specializations, respectively "informations," 
which agree on both sides of (3.3), in order to show that both sides are equal. 

In the same way as this is done in Lemma 1, one can show that, if one assumes the 
truth of (3.3) with replaced by A — 1, this implies that (3.3) is true if we specialize 
xi to X2. Similarly to the proof of Lemma 1, it is easy to see that both sides of (3.3) 
are symmetric in the variables xi,X2, ■ ■ ■ ,X2n+i, and also that they remain invariant up 
to an overall multiplicative sign if we replace xi by x^^. This means that identity (3.3) 
holds for all specializations of xi to one of X2, ^2 ^, ^3, Xg ^, . . . , X2Ar+i, These are 

4A specializations. We still need 3 additional specializations, respectively "informations," 
which agree on both sides of (3.3). 

We get two more specializations by observing that both sides of (3.3) vanish for xi = 
±1. For the left-hand side this is obvious because of the factors R(^A,A~^^ respectively 
R[A'^, (A*^)"^) appearing in the summand. On the right-hand side, the summands indexed 
by A and AAjl} cancel each other for xi = ±1. 

Finally, the coefficients of x^^+i on the left-hand and right-hand sides of (3.3) are re- 
spectively 



J2 {-l)^V{A)V{A-^) V{A'')V{{A'')-^) R{A, A'^) R{A'', {A'')-'^) 



AC[2N+1]\{1} 
\A\=N 



and 



E 

AC[2N+1]\{1} 



{-1)^V{A)V{A-^) V{A^)V{{A'')-^) R{A, (A^)"^) A'^) , 



where, in both cases, A'^ now denotes the complement of A in [2A-|- 1]\{1}. The equality of 
these two sums was established in Lemma 1. This completes the proof of the lemma. □ 

4. Proofs of theorems. This section is devoted to the proofs of Theorems 1 and 2. 
The idea is to substitute the determinantal definitions (2.1)-(2.6) of the characters into 
(1.1) respectively (1.2), expand the determinants in the numerators by using Laplace ex- 
pansion respectively linearity of the determinant in the rows, evaluate the resulting simpler 
determinants by means of one of the Weyl denominator formulas, and reduce the resulting 
expressions. By collecting appropriate terms, it is then seen that both identities result from 
Lemma 1 in the preceding section. 

Proof of Theorem 1. We start with the left-hand side of (1.1). By (2.1), we have 



2mx{t<n)+2n-t l<h<n 



. ,det I -(2mx(t<n)+2n-t) , i ^ 7 ^ o 



^((2m)") (^1) ^1 1X21X2 i---i-^niX^ ) 



y([n])F([n]-i)i?([n],[n]-i) 

(4.1) 

where x(5) = 1 if 5 is true and x(5) = otherwise. Here, we used the evaluation of the 
Vandermonde determinant (the Weyl denominator formula for type A; cf. [4, p. 400 and 
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Lemma 24.3]) in the denominator. We now do a Laplace expansion of the determinant 
along the first n columns. Abbreviating the denominator on the right-hand side of (4.1) by 
Di(n), this leads to 

'S((2m)") (^1; ^1 ^^2:^2 , . . . , X^, ) 

= — V (-l)(^<^6^") + (^''6B0+"l-S|-("^')detMiM, m-detM2(A^S^), 
Di{n) ^-^ o^ 

^ ^ A,BC[n] (4.2) 
|A| + |B|=n 

where Mi[A, B) is the n x n matrix 

^2m+2n-t h E A\ 



and M2{A'^, B'^) is the nx n matrix 

-in-t) 



xl'* he A" 



with A'^ denoting the complement of A in [n], and an analogous meaning for B'^. All 
determinants in (4.2) are Vandermonde determinants, except for some trivial factors which 
can be taken out of the rows of Mi{A^B) respectively M2(^,-B), and can therefore be 
evaluated in product form. If we substitute the corresponding results, we obtain 

^^^^> A,BC[n] ^aeA ^ ^ beB ^ 

\A\ + \B\=n 

■ ViA)V{B-^)RiA, B-^)V{A'')V{{B^)-^) R{A'', {B^)'^) . (4.3) 
Next we turn to the right-hand side of (1.1). By (2.2), we have 

det 

/ \ l</i,t<n 

l<n,i<n 

det 

l<h,t<n 



vi[n]) n^i x-^^^{xy' - x;'/') ni<.<,<n(^. - ^7') ' (4.4) 

where we have used the Weyl denominator formula (2.3). We now use linearity of the 
determinant in the rows. Abbreviating the denominator on the right-hand side of (4.4) by 
D2{n), this leads to 

S0^rnr^){xuX2,...,Xn) = ("1) ^ ^"^^ "^''^^^'^ det M3(^), (4.5) 
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where Ms{A) is the n x n matrix 



X 



-X 



m+n— 
h 

-(m+n— 



h 



he A 
he A" 



with A'^ denoting the complement of A in [n], as before. AU determinants in (4.5) are Van- 
dermonde determinants, except for some trivial factors which can be taken out of the rows of 
M^{A), and can therefore be evaluated in product form. If we substitute the corresponding 
results, we obtain 



{xi,X2,...,Xn) 

=i^EM)(--'-«')-(n^^)"'^*(n^.i"^* 

^ ^ AC[n] ^aeA ^ ^ a6A<= ^ 

•V(A)V((A'=)-i)i?(A,(A'=)-i), (4.6) 



whence the product of the two characters on the right-hand side of (1.1) can be expanded 
in the form 



so(^n)(a;i, X2-, • • • ,a;„) so(^r.)(-a;i, -x-i-, ■■ ., -Xn) 



Di(n) 



n-.) (n-.-') (n-») (n-.-' 

aeA ^ ^aeA'' ^ ^ beB ^ ^beB'= 

■ V{A)V {{A")-^) R{A, {A")-^) V{B)V ((S=)"') R{B, (S^)"') . (4.7) 



We now fix disjoint subsets A' and B' of [n] and extract the coefficient of 



n 

■ aeA' 



2m+n 



n 

■ beB' 



2m+n 



Xi 



(4.8) 



in (4.3). (When we here say "extract the coefficient of (4.8)," we treat "m" as if it were a 
formal variable. That is, in the sum on the right-hand side of (4.3), terms must be expanded, 
from each term which results from the expansion we factor out the product (4.8), and, if 
whatever remains is independent of m, it contributes to the coefficient.) This coefficient is the 
subsum of the sum on the right-hand side of (4.3) consisting of the summands corresponding 
to subsets A and B of [n] with A' = A\B and B' = B\A such that |^| -|- |5| = n. We note 
at this point that this implies that C :— [n]\{A' U B') must have even cardinality. We let 
A" be the intersection A" = An B, so that A = A'UA" and B B'UA" (with U denoting 
disjoint union), and we denote the complement of A" in C by {A"y. Since \A\ + \B\ — n, 
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we have \A"\ = \ {A"y\. Using the above notation, then, after some manipulation, the above 
described subsum can be rewritten as 

-l—(-l)^\B'\+^\c\v(A')V(B')V((AT^)V(m^^ 
Di[n) 

X R{A', (B')-^) R{B', (A')-^) RiC,A') R{C, (A')-^) R{C-\B') R{C-\ (B')-^) 

X V{A")V{{A")-^) R{A", {A")-^) V{{A"y)V{{{A"y)-^) R{{A"y, ((A")')"') , 

A"<zc (4.9) 

|A"| = i|C| 

where we used variations of the resultant notation . . ), namely 

R{A,B) := H llixa-Xk), 

a&AbeB 

R{A-\B) := \[X[{x-' - 

aeAb€B 

and 

R{A-\B-') :=llIl(^a'-H') 

aeA beB 

Now we turn our attention to (4.7). First of all, we observe that by exchanging the roles 
of A and B in the summand on the right-hand side of (4.7), the summand does not change 
except for a sign of (— l)!"^!"!^'. Thus, all summands corresponding to sets A and B whose 
cardinalities do not have the same parity cancel each other. We can therefore restrict the 
sum in (4.7) to subsets A and B of [n] with |^| = \B\ (mod 2). Consequently, if we extract 
the coefficient of (4.8) in (4.7), then, after some manipulation, we obtain 

X R{A', (S')-') R{B', {A')-^) R{C,A') R{C, {A')-^) R{C-\B') R{C-\ (S')"') 

X J2 v{A")v{{A")-^) r{a", {{A"y)-^) v{{A"y) v{{{A"y)-^) R{{A"y, {a")-^) . 

A"cc (4.10) 

Here, with A and B as in (4.7), the meanings of A', B', A", and C are A' — A n B, 
B' = [n]\{A U S), A" = A\B, C = [n\\A' U S', and {A")" is the complement of A" in C, 
so that A = A' OA" and B = A'U{A"y. Because of the restriction |^| = |S| (mod 2), again 
C must have even cardinality. Clearly, clearing the factors common to (4.9) and (4.10), the 
equality of (4.9) and (4.10) is a direct consequence of (3.1). This completes the proof of the 
theorem. □ 

Proof of Theorem 2. As we have shown when we rewrote Theorems 1 and 2 uniformly as 
(1.5), Theorem 2 results from the proof of Theorem 1 by "replacing m by m + Since, 
in the proof of Theorem 1, it is nowhere used that m is an integer, and since, in fact, m is 
treated there as a formal variable. Theorem 2 follows immediately. □ 
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5. Combinatorial interpretations. The purpose of this section is to give combinatorial 
interpretations of Theorems 1 and 2. As we aheady said in the introduction, these interpre- 
tations were at the origin of this work, which suggested Theorems 1 and 2 in the first place. 
They involve rhombus tilings, respectively plane partitions. When we speak of a rhombus 
tiling of some region, we always mean a tiling of the region by unit rhombi with angles of 60° 
and 120°. Examples of such tilings can be found in Figures 1-5. (Dotted lines and shadings 
should be ignored at this point.) We shall not recall the relevant plane partition definitions 
here, but instead refer the reader to [2] , and to [6] for explanations on the relation between 
plane partitions and rhombus tilings of hexagons. 

We claim that, by specializing xi = X2 = ••• = Xn = 1 in (1.1), we obtain the combina- 
torial factorization 



where PP{2m, n, n) denotes the number of plane partitions contained in the (2m) x n x 

n box (or, equivalently, the number of rhombus tilings of a hexagon with side lengths 
2m, n, n, 2m, n, n; see Figure 1 for an example in which m = 2 and n = 3), SPP{2m, n, n) 
denotes the number of symmetric plane partitions contained in the (2m) x n x n box (or, 
equivalently, the number of rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n 
which are symmetric with respect to the vertical symmetry axis of the hexagon; see Figure 2 
for an example in which m = 2 and n = 3), and TCPP{2m,n,n) denotes the number of 
transpose complementary plane partitions contained in the (2m) xnxn box (or, equivalently, 
the number of rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n which are 
symmetric with respect to the horizontal symmetry axis of the hexagon; see Figure 3. a for 
an example in which m = n = 3; the dotted lines should be ignored for the moment).^ 



PP(2m, n, n) = SPP{2m, n, n) ■ TCPP{2m, n, n), 



(5.1) 




A rhombus tiling of a hexagon 



Figure 1 



■"^Clearly, the factorization (5.1) could be readily verified directly by using the known product formulas 

for PP{2m,n,n), SPP{2m,n,n), and TCPP{2m,n,n) (cf. [2]). However, the point here is that it is a 
consequence of the more general factorization (1.1) featuring a Schur function and odd orthogonal characters. 
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We start with the left-hand side of (5.1). The fact that the speciahzed Schur function 
S((2m)")(l) 1) • • • ) 1) (with 2n occurrences of 1 in the argument) is equal to the number of 
plane partitions in the (2m) x n x n box, and that this is equal to the number of rhom- 
bus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n, is well-known (cf. [6] and [10, 
Sec. 7.21]). 




A vertically symmetric rhombus tiling of a hexagon 
Figure 2 



It is also well-known (see [2, Sec. 4.3] or [8, Ch. I, Sec. 5, Ex. 15-17]) that the number of 
symmetric plane partitions contained in the (2m) x n x n box is equal to so(^n)(l, 1, . . . , 1) 
(with n occurrences of 1 in the argument). 

The argument which explains that so(TOn)(— 1, —1, . . . , —1) (with n occurrences of —1 in 
the argument) counts transpose complementary plane partitions contained in the (2m) xnxn 
box (up to sign) is more elaborate. We begin by setting Xi = —q^~^ in (2.2) with N = n 
and A = (m"^), to obtain 

det (^qih-l){m+n-t+^) _,_ ^-(/i-l)(m+n-t+i)^ 
l<h,t<n 

The determinants on the right-hand side can be evaluated by means of (2.7). (This is seen 
by replacing hhy n + 1 — h and then interchanging the roles of h and t in the determinants 
above.) If we subsequently let q tend to 1, then we obtain 

«<>(„»,(-l,-l,...,-l) = (-l) n 2n + l-h-t ■ 

l<h<t<n 

On the other hand, the product on the right-hand side is equal to a specialized symplectic 
character. Namely, by specializing A = (m^), Xi — q'^, i — 1,2, . . . ,N, in (2.4), using the 
identity (2.5), and finally letting q tend to 1, it is seen that 

, TT 2m + 2N + 3-h-t 

.P(^.)(l,l,...,l)= [[ 2N + 3-h-t 

l<h<t<N+l 
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(with N occurrences of 1 in the argument). Thus, for A?" = n — 1, we obtain that 

-1, . . . , -1) = (-1)-'^ sp^^r.-,^{l, !,...,!). 




a. A horizontally symmetric rhombus tiling of a hexagon b. The corresponding path family 




c. Orthogonal lattice paths d. Shorter non-intersecting lattice paths 

Figure 3 

To conclude the argument, it is easy to see using famihes of m lattice paths starting 
at the mid-points of the top-most m edges along the left vertical side of length 2m of the 
hexagon with side lengths 2m, n, n, 2m, n, n and ending at the mid-points of the top-most 
m edges along the right vertical side of length 2m of the hexagon, the paths "following the 
rhombi" of the tiling (and, thus, staying necessarily in the upper half of the hexagon; see 
Figure 3.a,b), that rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n which 
are symmetric with respect to the horizontal symmetry axis of the hexagon are in bijection 
with families (Pi, P2, . . . , Pm) of non-intersecting lattice paths consisting of horizontal and 
vertical unit steps, the path Pi starting at — 1) and ending at {n — i,n + i ~ 1), 

i = 1, 2, . . . , m, all paths never passing below the line y — x + 1 (see Figure 3.c for the path 
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family resulting from the one in Figure 3.b by deforming the paths to orthogonal ones; as 
usual, the term "non-intersecting" means that no two paths in the family have a common 
point). Because of the boundary y = x + 1 and the condition that paths are non-intersecting, 
the initial step of any path in such a family must be a vertical step while the final step of 
any path must be a horizontal step. Thus, our rhombus tilings are in bijection with families 
(Pi, P2, ■ ■ ■ , Pm) of non- intersecting lattice paths consisting of horizontal and vertical unit 
steps, the path Pi starting at (— z, i) and ending at (n — 1 — i, n — 1 + i) , i = 1,2, ... ,171, all 
paths never passing below the line y = x+1 (see Figure 3.d). It is known (see [3, Sec. 5]) that 
these families of non-intersecting lattice paths are counted by sp(^^„-i)(l, !,...,!) (with n 
occurrences of 1 in the argument). Thus, | so(^«)(— 1, —1, . . . , — 1)| counts indeed rhombus 
tilings of a hexagon with side lengths 2m, n, n, 2m, n, n which are symmetric with respect to 
the horizontal symmetry axis of the hexagon, respectively transpose complementary plane 
partitions in the (2m) x n x n box. 

For Theorem 2 we are also able to provide a combinatorial interpretation in the context 
of rhombus tilings. However, it may be less convincing. 

We specialize a;i = 0:2 = • • • = x„ = 1 in (1.2). Clearly, S((2Tn+i)") (I7 17 • • ■ 7 1) (with 
2n occurrences of 1 in the argument) counts plane partitions in the (2m + 1) x n x n box, 
respectively rhombus tilings of a hexagon with side lengths 2m -|- l,n,n, 2m + l,n,n. 




A horizontally symmetric rhombus tiling of a hexagon with two missing triangles 

Figure 4 



Moreover, the argument above shows that sp(^„)(l, !,...,!) (with n occurrences of 1 in 
the argument) counts rhombus tilings of a hexagon with side lengths 2m + l, n, n, 2m + 1, n, n 
from which two unit triangles have been removed on the left and the right end of the 
horizontal symmetry axis of the hexagon, the tilings being symmetric with respect to this 
axis (see Figure 4 for an example in which m = 2 and n — 3). Alternatively, this is also 
the number of horizontally symmetric rhombus tilings of a (full) hexagon with side lengths 
2m, n + l,n + 1, 2m, n + l,n + 1 (compare with Figure 3. a). 

In order to find a combinatorial interpretation of o'^'^^^-^n^ (1, 1, • • • > 1) (with n occurrences 
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of 1 in the argument), we start with the decomposition 

^(('m+l)")!!' 1, . . . , 1) = SO(('^'^i)„)(l, 1, . . . , 1) + SO^^'^'^i^n-i 1, . . . , 1), 

where 

det (a;J*+-* + a;-(^'+"-*))+ det - 

p,,p„. s l<h,t<n l<h,t<n " 

= 

l<.n,t<n 

is an irreducible character of S02ni^) (and its spin covering group; see [4, (24.40)]). Here, 
A = (Ai, A2, . . . , A„) is a non- increasing sequence of (possibly negative) integers or half- 
integers with A„_i > |A^|. It was proved in [1] (see [5] for a common generalization) that, 
for all non-negative integers or half-integers c, we have 

S0l'^u'^{xi,X2,...,Xn)^{xiX2---Xny- Sj. (xi , X2, . . . , X^) (5.2) 

iyC((2c)") 
oddcols (((2c)")/i/)=0 

and 

'V"-i,-c) 



S0l^^-i_^^{xi,X2,...,Xn)^{xiX2---Xn) " ■ ^ S^{xi, X2, ■ ■ ■ , Xn) , (5.3) 

i/C((2c)") 
oddcols (((2c)")/i/) =2c 

where oddcols(((2c)'^)/z^) denotes the number of odd columns of the skew diagram (cf. [8, 
p. 4]) {{2c)'') /u. Thus, we have 

«((T+i)")(l,l, •••,!)= ^.(1,1, •••,!), 

i/C((2m+2)") 

where is taken over all diagrams u with the property that {{2m + 2)^)/u consists either 
of only even columns or of only odd columns. (Both the even orthogonal character on the 
left-hand side and the Schur functions on the right-hand side contain n occurrences of 1 in 
their arguments.) 

Given a shape v contained in ((2m-|-2)"'), it is well-known (cf. [3, Sec. 4] or [9, Ch. 4]) that 
Siy{l, 1, . . . , 1) counts families (Pi, . . . , P2m+2) of non-intersecting lattice paths consisting of 
horizontal and vertical unit steps, where the path Pi runs from (— z, i) to {i/'^ — i,n — u'^^ + i), 
i = 1,2,..., 2m + 2. (Here, u' denotes the partition conjugate to u; cf. [8, p. 2]). On the 
other hand, in a similar way as above, such families of non-intersecting lattice paths (if v 
is allowed to be any partition) are in bijection with rhombus tilings of a hexagon with side 
lengths 2m + 2, n, n, 2m + 2, n, n which are symmetric with respect to the vertical symmetry 
axis of the hexagon (see Figure 5 for an example in which m = 2, n = 4, u = (5, 5, 2, 2)). 
The property that ((2m-|-2)"^)/z^ consists either of only even columns or of only odd columns 
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a. A vertically symmetric rhombus tiling of a hexagon b. Non-intersecting lattice paths 

Figure 5 

translates into the property that, in the corresponding rhombus tihngs, chains of successive 
horizontally oriented rhombi along the vertical symmetry axis of the hexagon can appear 
in the interior of the hexagon only if they have even length. (By definition, a "chain of 
horizontally oriented rhombi along the vertical symmetry axis of the hexagon" is a set of 
horizontally oriented rhombi sitting on the vertical symmetry axis which, together, form a 
topologically connected set. For a chain, to be in the interior, means that none of the rhombi 
of the chain touches the boundary of the hexagon. In Figure 5. a, there are two such chains. 
They consist of the two contiguous strings of grey shaded rhombi, respectively.) 

In summary. Theorem 2, when specialized to xi = X2 = ■ ■ ■ = Xn = 1, can be in- 
terpreted combinatorially as follows: the term S((2m-i-i)")(l) 1) • • • > 1) on the left-hand side 
counts rhombus tilings of a hexagon with side lengths 2m + l,n,n, 2m + 1, n, n. The term 
5P(-^„)(1, 1, . . . , 1) counts horizontally symmetric rhombus tilings of a hexagon with side 
lengths 2m + 1, n, n, 2m + 1, n, n from which two unit triangles have been removed on the 
left and the right end of the horizontal symmetry axis of the hexagon. Finally, the term 
''((m+i)") (-'-'-'-'•••' -'-) counts vertically symmetric rhombus tilings of a hexagon with side 
lengths 2m -|- 2, n, n, 2m + 2,n,n with the property that chains of successive horizontally 
oriented rhombi along the vertical symmetry axis of the hexagon can appear in the interior 
of the hexagon only if they have even length. We remark that these rhombus tilings could 
equivalently be seen as symmetric plane partitions in a (2m+2) xnxn box in which "central 
terraces" (that is, horizontal levels situated along the plane of symmetry) must have even 
length, except at height and at height 2m + 2. In other words, this specialization yields 
the identity 

PP{2m + 1, n, n) = TCPP{2m, n + 1, n + 1) SPP'{2m + 2, n, n), (5.4) 
where the * indicates that only those symmetric plane partitions are counted which satisfy 
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the "even central terraces in the interior" condition described above. In particular, by 
specializing Xh = q'^~^ in (2.6), using (2.7) for evaluating the two determinants in (2.6), and 
then letting q tend to 1, we obtain that 

SPI-{2m,n,n) = 2 J] '"^t-T-V ' ^'''^ 

l<h<t<n 

6. More factorization theorems. In this final section, we present two further factoriza- 
tion theorems similar to those of Theorems 1 and 2, which have been hinted at to us by Ron 
King. Since these theorems can be proved in a manner very similar to the one which led to 
our proofs of Theorems 1 and 2, we content ourselves with giving only sketches of proofs, 
leaving the (easily filled in) details to the reader. 

Theorem 3. For any non-negative integers m and n, we have 

S((2m+l)")(2^1) ^ ^X2tX2^ T ■ ■ ■ iXniX^^) + S((2m+l)"-i) (^^l; , X2, , ■ ■ ■ , Xn, X^^) 

= {-1)'^'^ S0(^rn+l)r^){xi,X2, . . . ,Xn) SO^rn^){-Xi, -X2, ■ ■ ■ , -Xn) ■ (6.1) 

Sketch of proof . We proceed in the same way as in the proof of Theorem 1. By comparison 
with (4.3), we see that the left-hand side of (6.1) is equal to 

-, / \ / \ / \ 2m+n+l / \ 2m+n+l 

1 ^ (_i)(E...«).(E...)...i«H»r)(nx.) (n^.-') 

^^^> A,BC[n] ^aeA ^ ^ beB ^ 

\A\ + \B\=n 

■ V{A)V{B-^)R{A,B-^)ViA'')V{iB'')-^) (B^)-^) 

■i / \ / \ / \ 2m+n+2 / \ 2m+n+2 

+7r^ E (-i)(E."")+(s..»')+"W-m(jj,\ (jj,-A 

^^^^ A,BC[n] ^aeA ^ ^ beB ^ 

\A\ + \B\^n-l 

■ V{A)V{B-^)R{A, S-i)l/(A^)y ((S^)-i) i?(A^ (B^)-^) , (6.2) 

with -Di(n) having the same meaning as in the proof of Theorem 1. On the other hand, 
from (4.6) we see that the right-hand side of (6.1) is equal to 



^-^)"^" y (_l)(E.e.«) + (E.e.'')-('T^)-('f') 

Di(n) ^ 



( n ( n -0 ( n 

• V{A)V {{A^Y^) R{A, {A")-'^) V{B)V ((S'^)"^) R{B, {B")-^) 
tin Y (_i)(E...a)+(E...6)-(i-n-('f') 

^ ' A,B<Z\n\ 

X 2m+2 / X 2m+2 , \ / 

VlA ( n n -0 ( n 

V{A)V ((A^)-') i?(A (A^)-^) V{B)V ((S^)-') R{B, (B^)-^) . (6.3) 
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We now fix disjoint subsets A' and B' of [n] and extract the coefficients of 

(\ 2m+n / \ 2m+n 

n ( n 

in (6.2) respectively in (6.3) (in the same sense as in Section 4). If + has the same 
parity as n, then in (6.2) only the first sum contributes, while in (6.3) it is only terms where 
the cardinality of the symmetric difference AI\B is even. One can then see in the same 
way as Theorem 1 followed from Lemma 1, that the equality of corresponding contributions 
follows from Lemma 2. On the other hand, if + has parity different from the parity 
of n, then in (6.2) only the second sum contributes, while in (6.3) it is only terms where the 
cardinality of the symmetric difference A/\B is odd. In this case, the corresponding equality 
follows from Lemma 3. □ 

Theorem 4. For any non-negative integers m and n, we have 

^((2m)") (^1; ^1 ; ^2; ^2 t ■ ■ ■ ■> ) ~1" '5((2m)"-i) (^1 1 ^1 t X2 :---:XmX^ ) 

= SP(^m^-){xi, X2,..., Xn) 0(^T)(xi, X2, ■ ■ - .Xn). (6.4) 

Sketch of proof. In a similar manner as we saw that the proof of Theorem 2 follows by 
"replacing m by m + |" in the proof of Theorem 1, the proof of Theorem 4 follows by 
"replacing m by m — |" in the proof of Theorem 3. □ 

Again, using (1.3) and (1.4), Theorems 3 and 4 allow for a uniform statement, namely as 

( n*^^i + ^i^^^) ) (s(M") ixi,X^'^,X2,X2^, ...,Xn, X~^) 

+ S(M"-i) (^1) J ^2, X2^ , ■ ■ ■ , Xn, X^^)^ 
= {Xi, X2,..., Xn) O^^^M )„) (^1' a^2, • • • , Xn). (6.5) 

In a similar vein as in Section 5, by specializing = 1, z = 1, 2, . . . , n, in Theorems 3 and 
4, we are able to derive potentially interesting combinatorial interpretations. If we perform 
these specializations in (6.1), then, by the combinatorial facts explained in Section 5, we 
obtain the identity 

PP{2m + 1, n, n) + PP{2m + 1, n - 1, n + 1) = SPP{2m + 2, n, n) ■ TCPP{2m, n, n), (6.6) 

while the same specialization in (6.4) yields 

PP{2m, n, n) + PP{2m, n - 1, n + 1) = TCPP{2m, n + 1, n + 1) • SPP*{2m, n, n), (6.7) 

where PP{A, B, C) denotes the number of plane partitions contained in the AxBxC box (or, 
equivalently, the number of rhombus tilings of a hexagon with side lengths A, B, C, A, B, C, 
SPP{M, N, N) and TCPP{M, N, N) have the same meaning as in Section 5, and where the 
* in SPP*{2m,n,n) means that the symmetric plane partitions in consideration satisfy the 
"even central terraces in the interior" condition explained at the end of Section 5. 

Clearly again, the factorizations (6.6) or (6.7) could be readily verified directly by using 
the product formulas for the combinatorial quantities involved. However, it would have been 
difficult to see that they exist without having first considered the factorization identities for 
classical group characters given by Theorems 3 and 4, respectively. 
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